Redd Sightability Study

0.0.1 Collaborators

Russ Thurow USDA Forest Service Rocky Mountain Research Station

Claire McGrath Natural Resources Specialist, Columbia Hydropower Branch at NOAA Fisheries, West Coast Region, riverbio@yahoo.com

Kevin See Biometrician, Biomark Inc, Boise, ID, Kevin.See@merck.com

1 Goals

Salmon redd counts are widespread method to estimate the number of returning adult spawners. However, despite its prevalence in the Northwest, the reliability of redd counts is unknown. This work is focused on developing a statistical model to estimate the observer error in redd surveys, using a variety of covariates related to the habitat and the observer. We described three types of observer error:

• omission rate, \(\omega\) (proprotion of redds available to be counted that were missed by the observer)
• commission rate, \(\eta\) (rate of redds counted by an observer that were not actually redds)
• net error, \(\gamma\) (ratio of observed redds to true redds). This was modeled using log(net error) as the reponse.

2 Methods

Possible covariates in each error model are shown in Table 2.1. To make comparisons with AICc, the random effects must be identical across all models. Therefore, we ensured that the random effect of year was added to any model that didn’t have it.

Table 2.1: Possible covariates included in each observer error model.

Type	Covariate	Air	Ground
Random	Reach	X	X
Random	Surveyor		X
Random	Year	X	X
Fixed	Alluvium	X	X
Fixed	AveBadCond	X
Fixed	AveCanopy	X	X
Fixed	AveDepth	X	X
Fixed	AveSunny	X
Fixed	AveWidth	X	X
Fixed	EscapeEst	X	X
Fixed	Experience3		X
Fixed	I(AveDepth^2)	X	X
Fixed	Lithology	X	X
Fixed	LYabund	X	X
Fixed	LYabund:PeakQ	X	X
Fixed	PeakQ	X	X
Fixed	redd_dens_obs	X	X
Fixed	Slope	X	X

All covariates were z-scored, and all models were fit using the glmer or lmer functions from the lme4 package (Bates et al. 2015) in R software (R Core Team 2019). The amount of variation explained by fixed and random effects was calculated using the methods of Nakagawa and Schielzeth (2013). Using estimated predictions of the rates for omission (\(\hat{\omega}\)), commission (\(\hat{\eta}\)) and net error (\(\hat{\gamma}\)), we predicted the number of actual redds by either dividing the observed counts, \(c\), by estimates of net error, or by multiplying the observed counts by 1 - estimated rate of commission, and then dividing by 1 - estimated rate of omission.

We performed a cross validation by dividing each survey type data into 10 training datasets where 10% of the data was withheld for testing, and then fitting the naive and best AICc model formulations to the remaining data, and then using those fits to predict the error rates and true number of redds for each survey in the year that had been withheld.

\[ \begin{aligned} redds_{ne} &= \frac{c}{\hat{\gamma}} \\ redds_{om} &= c * \frac{1 - \hat{\eta}}{1 - \hat{\omega}} \end{aligned} \]

The observed error rates are showin in Figure 2.1.

Figure 2.1: Observed error rates.

3 Results

3.1 Model Coefficients

The model coefficients of the full, best (by AICc) and model averaged models are shown in Table 3.1 and 3.2.

Table 3.1: Estimated coefficients for ground observer error models.

Survey	Resp	Covariate	avg	best	full
Ground	Com	(Intercept)	-1.603	-1.611	-1.611
Ground	Com	AlluviumN	0.588	0.588	0.588
Ground	Com	AveCanopy	0.040	0.040	0.040
Ground	Com	AveDepth	0.170	0.170	0.170
Ground	Com	AveWidth	-0.143	-0.143	-0.143
Ground	Com	EscapeEst	-0.314	-0.314	-0.314
Ground	Com	Experience3.L	-0.242	-0.242	-0.242
Ground	Com	Experience3.Q	0.275	0.275	0.275
Ground	Com	I(AveDepth^2)	0.051	0.051	0.051
Ground	Com	Lithology2	0.389	0.389	0.389
Ground	Com	Lithology3	0.328	0.328	0.328
Ground	Com	Lithology5	0.324	0.324	0.324
Ground	Com	LYabund	-0.225	-0.225	-0.225
Ground	Com	LYabund:PeakQ	-0.741	-0.741	-0.741
Ground	Com	PeakQ	-0.683	-0.683	-0.683
Ground	Com	redd_dens_obs	0.444	0.445	0.445
Ground	Com	Slope	0.052	0.052	0.052
Ground	Net	(Intercept)	-0.258	-0.249	-0.845
Ground	Net	AlluviumN	-0.184		0.257
Ground	Net	AveCanopy	-0.041		0.023
Ground	Net	AveDepth	0.066		0.075
Ground	Net	AveWidth	-0.083		-0.028
Ground	Net	EscapeEst	-0.080		-0.080
Ground	Net	Experience3.L	0.318		0.294
Ground	Net	Experience3.Q	-0.169		-0.180
Ground	Net	I(AveDepth^2)	0.008		-0.007
Ground	Net	Lithology2	0.345		0.551
Ground	Net	Lithology3	0.251		0.455
Ground	Net	Lithology5	0.422		0.556
Ground	Net	LYabund	0.092		-0.036
Ground	Net	LYabund:PeakQ	0.040		-0.177
Ground	Net	PeakQ	-0.031		-0.163
Ground	Net	redd_dens_obs	0.143	0.143	0.189
Ground	Net	Slope	-0.047		-0.055
Ground	Omi	(Intercept)	0.301	0.301	0.301
Ground	Omi	AlluviumN	-0.049	-0.049	-0.049
Ground	Omi	AveCanopy	-0.027	-0.027	-0.027
Ground	Omi	AveDepth	0.107	0.107	0.107
Ground	Omi	AveWidth	-0.176	-0.176	-0.176
Ground	Omi	EscapeEst	-0.029	-0.029	-0.029
Ground	Omi	Experience3.L	-0.599	-0.599	-0.599
Ground	Omi	Experience3.Q	0.608	0.608	0.608
Ground	Omi	I(AveDepth^2)	-0.023	-0.023	-0.023
Ground	Omi	Lithology2	-0.745	-0.745	-0.745
Ground	Omi	Lithology3	-0.704	-0.704	-0.704
Ground	Omi	Lithology5	-0.741	-0.741	-0.741
Ground	Omi	LYabund	-0.186	-0.186	-0.186
Ground	Omi	LYabund:PeakQ	-0.339	-0.339	-0.339
Ground	Omi	PeakQ	-0.053	-0.053	-0.053
Ground	Omi	redd_dens_obs	-0.126	-0.126	-0.126
Ground	Omi	Slope	0.336	0.336	0.336

Table 3.2: Estimated coefficients for air observer error models.

Survey	Resp	Covariate	avg	best	full
Air	Com	(Intercept)	-1.355	-1.327	-4.067
Air	Com	AlluviumN	0.562		1.491
Air	Com	AveBadCond	0.003		0.011
Air	Com	AveCanopy	-0.151		0.291
Air	Com	AveDepth	0.006		0.175
Air	Com	AveSunny	0.345	0.345
Air	Com	AveWidth	0.289		0.347
Air	Com	EscapeEst	-0.291		-0.291
Air	Com	I(AveDepth^2)	-0.035		-0.053
Air	Com	Lithology2	1.533		2.804
Air	Com	Lithology3	0.550		0.559
Air	Com	Lithology5	1.305		2.863
Air	Com	LYabund	0.442		0.431
Air	Com	LYabund:PeakQ	0.314		0.188
Air	Com	PeakQ	-0.324		-0.488
Air	Com	redd_dens_obs	0.069		-0.034
Air	Com	Slope	-0.179		-0.370
Air	Net	(Intercept)	-0.195	-0.183	-1.196
Air	Net	AlluviumN	-0.222		0.405
Air	Net	AveBadCond	0.003		0.027
Air	Net	AveCanopy	-0.148		0.003
Air	Net	AveDepth	0.078		0.118
Air	Net	AveSunny	0.220	0.220
Air	Net	AveWidth	0.128		0.112
Air	Net	EscapeEst	-0.089		-0.089
Air	Net	I(AveDepth^2)	-0.016		-0.023
Air	Net	Lithology2	0.563		0.938
Air	Net	Lithology3	0.199		0.318
Air	Net	Lithology5	0.529		1.069
Air	Net	LYabund	0.134		0.108
Air	Net	LYabund:PeakQ	0.158		0.086
Air	Net	PeakQ	-0.084		-0.158
Air	Net	redd_dens_obs	0.109		0.058
Air	Net	Slope	-0.099		-0.095
Air	Omi	(Intercept)	-0.528	-0.528	0.050
Air	Omi	AlluviumN	-0.082		-0.082
Air	Omi	AveBadCond	-0.007		-0.007
Air	Omi	AveCanopy	0.079		0.079
Air	Omi	AveDepth	0.006		0.006
Air	Omi	AveSunny	-0.257
Air	Omi	AveWidth	-0.131		-0.131
Air	Omi	EscapeEst	0.151		0.151
Air	Omi	I(AveDepth^2)	-0.004		-0.004
Air	Omi	Lithology2	-0.921		-0.921
Air	Omi	Lithology3	-0.934		-0.934
Air	Omi	Lithology5	-0.280		-0.280
Air	Omi	LYabund	0.185		0.185
Air	Omi	LYabund:PeakQ	0.132		0.132
Air	Omi	PeakQ	0.260		0.260
Air	Omi	redd_dens_obs	-0.546	-0.546	-0.560
Air	Omi	Slope	0.159		0.159

3.2 Ground Surveys

The relative importance of each covariate in each model is shown in Figure 3.1, while the amount of the variance explained by fixed and random effects in the best AICc model is shown in Figure 3.2. Observed versus predicted rate plots are shown in Figures 3.3, 3.5 and 3.7.

Figure 3.1: Relative importance of each covariate in ground-based observer error models

Figure 3.2: How much variance in the model response is explained by the fixed and random effects in the best AICc model.

3.2.1 Omission

Figure 3.3: Observed versus predicted rates of omission using model averaged predictions, the single best model, and the naive model (only random effects).

Figure 3.4: Correlations between observed omission rates and three model predictions (model averaged, single best and naive).

3.2.2 Commission

Figure 3.5: Observed versus predicted rates of commission using model averaged predictions, the single best model, and the naive model (only random effects).

Figure 3.6: Correlations between observed commission rates and three model predictions (model averaged, single best and naive).

3.2.3 Net Error

Figure 3.7: Observed versus predicted rates of net error using model averaged predictions, the single best model, and the naive model (only random effects).

Figure 3.8: Correlations between observed net error rates and three model predictions (model averaged, single best and naive).

3.2.4 Leave-One-Out Cross Validation

3.2.4.1 Rate Estimates

We examined the bias in estimates rates, using both the best (by AICc) model and the naive model (only random effects) (Figure 3.9).

Figure 3.9: Bias in predicted ground error rates from cross validation results.

3.2.4.2 Redd Estimates

For ground-based surveys, both methods provided fairly unbiased estimates of the true number of redds (Figure 3.10), although the omission/commision models had slightly higher absolute and relative bias (Table 3.3).

Figure 3.10: Boxplots of absolute and relative bias for each type of predictive model.

Table 3.3: Summary statistics of predictions of total redds from leave-one-out cross validation using the net error and the omission/commission models.

Model	Median # Obs. Redds	Median # True Redds	Median Adjustment	Median Abs. Bias	Median Rel. Bias (%)	RMSE
Best Net Error	36	38	3.9	0.1	0.3	18.8
Best Omis / Comm Error	36	38	4.8	0.6	1.0	16.2
Naive Net Error	36	38	3.3	-1.0	-2.1	19.5
Naive Omis / Comm Error	36	38	4.1	-0.2	-0.6	18.2
Observed	36	38		-2.0	-8.0	18.9

Figure 3.11: Observed number of true redds vs. leave-one-out cross validated predicted redds based on either the best AICc or naive versions of the net error or omission/commission models. Dashed line is the 1-1 line, while solid line with gray error ribbon is the best fit linear model to these data.

3.3 Air Surveys

The relative importance of each covariate in each model is shown in Figures 3.12, while the amount of the variance explained by fixed and random effects in the best AICc model is shown in Figure 3.13. Observed versus predicted rate plots are shown in Figures 3.14, 3.16 and 3.18.

Figure 3.12: Relative importance of each covariate in ground-based observer error models

Figure 3.13: How much variance in the model response is explained by the fixed and random effects in the best AICc model.

3.3.1 Omission

Figure 3.14: Observed versus predicted rates of omission using model averaged predictions, the single best model, and the naive model (only random effects).

Figure 3.15: Correlations between observed omission rates and three model predictions (model averaged, single best and naive).

3.3.2 Commission

Figure 3.16: Observed versus predicted rates of commission using model averaged predictions, the single best model, and the naive model (only random effects).

Figure 3.17: Correlations between observed commission rates and three model predictions (model averaged, single best and naive).

3.3.3 Net Error

Figure 3.18: Observed versus predicted rates of net error using model averaged predictions, the single best model, and the naive model (only random effects).

Figure 3.19: Correlations between observed net error rates and three model predictions (model averaged, single best and naive).

3.3.4 Leave-One-Out Cross Validation

3.3.4.1 Rate Estimates

We examined the bias in estimates rates, using both the best (by AICc) model and the naive model (only random effects) (Figure 3.20).

Figure 3.20: Bias in predicted air error rates from cross validation results.

3.3.4.2 Redd Estimates

For air-based surveys, both methods provided estimates of the true number of redds that were biased high (Figure 3.21). However, the net error models had lower absolute and relative bias, as well as a smaller root squared mean error (RMSE) (Table ??).

Figure 3.21: Boxplots of absolute and relative bias for each type of predictive model.

Table 3.4: Summary statistics of predictions of total redds from leave-one-out cross validation using the net error and the omission/commission models.

Model	Median # Obs. Redds	Median # True Redds	Median Adjustment	Median Abs. Bias	Median Rel. Bias (%)	RMSE
Best Net Error	32	38	6.4	1.7	9.5	24.5
Best Omis / Comm Error	32	38	10.3	0.8	1.3	24.2
Naive Net Error	32	38	5.8	-0.5	-3.2	26.3
Naive Omis / Comm Error	32	38	7.8	-1.2	-3.1	27.7
Observed	32	38		-6.0	-18.2	28.7

Figure 3.22: Observed number of true redds vs. leave-one-out cross validated predicted redds based on either the best AICc or naive versions of the net error or omission/commission models. Dashed line is the 1-1 line, while solid line with gray error ribbon is the best fit linear model to these data.

4 Discussion

For both ground- and air-based surveys, across all three types of models, the best AICc model predictions and the model averaged predictions were very similar, so although I didn’t include model averaged predictions in my comparisons, I expect them to be very similar to the best model. I chose to use the best AICc model because it would be simplier.

Models for both types of surveys tended to provide unbiased predictions of the number of redds in the stream. Ground-based surveys required slightly less of an adjustment (in either direction) than air-based surveys, but there was certainly an adjustment in both cases, highlighting the need for such observer error models. The median absolute bias for both types of surveys and for both modeling approaches was less than one redd, compared to an undercount of 2 redds for ground-based surveys, and 6 for air-based surveys.

For the ground-based surveys, the most important covariates to explain net error were observed redd density and observer experience, whereas most of the possible covariates had similar importance for omission and commission models. However, the predictions of total redds are almost identical regardless of whether one uses the net error or omission / commission models, or the best model by AICc, or the naive model (only random effects). This suggests that the fixed effects covariates included in this study are not explaining much of the variation in error rates, but that there is information in the random effects of reach and surveyor sufficient to make unbiased predictions. However, those random effects cannot be easily carried on to another study area, or even another year, which limits their usefulness as a predictive model.

For the air-based surveys, AveSunny figured prominently in net error and commission models, whereas observed redd density was clearly the most important for explaining omission errors. The net error and omission / commission models, either the best AICc or the naive versions, all made very similar unbiased predictions, although they appear to slightly under-predict at higher number of redds.

The fact that the predictions from the naive model (random effects only) are highly correlated with the best and model averaged versions for all the various error models, except for net errors for air surveys, suggests that the fixed effects are not explaining much of the observed variation in error rates. This is also seen in Figure 3.2 and 3.13. The root mean squared error (RMSE) of predictions is slightly smaller when using the best AICc model compared to the naive model, so there is some benefit to those fixed effects.

References

Bates, D., Mächler, M., Bolker, B., and Walker, S. 2015. Fitting Linear Mixed-Effects Models Using lme4. Journal of Statistical Software, 67(1): 1–48.

Nakagawa, S., and Schielzeth, H. 2013. A general and simple method for obtaining r2 from generalized linear mixed-effects models. Methods in Ecology and Evolution 4(2): 133–142. Wiley Online Library.

R Core Team. 2019. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria.

---

1. Biometrician, Biomark, Inc., Kevin.See@merck.com↩︎

2. Natural Resources Specialist, Columbia Hydropower Branch at NOAA Fisheries, West Coast Region, riverbio@yahoo.com↩︎

3. USDA Forest Service Rocky Mountain Research Station↩︎